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ABSTRACT 

The Florida Successful Schools Project, a 
collaborative effort of the State Department of Education, school 
district Chapter 1 programs, and the Technical Assistance Centers in 
Atlanta, is designed to help the Florida schools that face the 
greatest challenges in helping students achieve in a safe and orderly 
environment. This report covers the first year of the project, which 
focused on three research perspectives: (1) a data base analysis of 
information about the schools; (2) staff and parent surveys based on 
effective school correlates; and (3) on-site observations and 
interviews. It found that schools with high concentrations of 
students from low-income families face tremendous challenges in 
meeting state educational goals for student performance, learning 
environment, and school safety. Nevertheless, some high poverty 
schools do better than state averages. These schools are 
characterized by higher student achievement, fewer minority students, 
more support staff, and lower suspension rates. The effective school 
correlates that provide the greatest contrasts between high and low 
achieving schools are safe and orderly learning environment, 
instructional leadership, and high expectations for students. 
Nineteen graphs and one figure present information about Florida 
schools. A separate document contains supporting technical papers. 
(Contains 22 references.) (SLD) 
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Chapter 1 



Successful Schools 



^ Pilot Project Report 

Executive Summary 



Part of the vision of Blueprint 2000 is that all schools will enable their students to 
achieve at the highest levels in a safe and orderly environment. The Florida 
Successful Schools project, a collaborative effort of staff from district Chapter 1 
programs, the Department of Education, and the Technical Assistance Centers in 
Atlanta, is specifically designed for those schools that face the greatest challenge in 
realizing that vision. 

The project began in September 1993, and is aimed at: 

Identifying characteristics that 
distinguish schools that exceed state 
averages on a majority of reading, 
math and writing assessments and 
also have 65% or more of their 
students receiving Free or Reduced 
Price meals; 



Sharing research findings that will 
support school improvement, 
especially in areas that have large 
numbers of children from low-income 
families; and 



Developing training and technical 
assistance strategies for schools that 
are operating or plan to operate 
Chapter 1 Schoolwide projects. 

This report covers the first year of the project which focused on three research 
perspectives: a data base analysis, using Florida School Reports and related 
information; staff and parent surveys based on effective school correlates; and on-site 
observations and interviews. Highlights of the findings from these three areas are: 




[ Schools with high concentrations of students from low-income 

i^nggg^t families face tremendous challenges in meeting state education 

'S Kt ii ^ SS goals for student performance, learning environment, and school 

safety. 




Despite the odds against them, some high poverty schools actually 
do better than state averages and even better than some low 
poverty schools. 



Attributes such as promotion, attendance. Limited English 
Proficiency, mobility, class size, regular program expenditures, or 
teachers with advanced degrees do not distinguish successful high 
poverty schools from similar, but lower achieving schools. 



as 
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The successful schools are distinguishable in terms of having higher 
student achievement, fewer minority students, more support staff, 
and lower suspension rates. 



Staff surveys describe three effective school correlates that provide 
the greatest contrast between higher and lower achieving schools: 
Safe and Orderly Learning Environment, Instructional Leadership and 
High Expectations for students. 




Results of parent surveys show that the only questions that clearly ' 
distinguish between higher and lower achieving schools have to do 
with safety and discipline. 



On-site observations and interviews reveal only a few distinguishing 
features between higher and lower achieving schools primarily in 
leadership styles and focus of the school's mission. 

The Florida Successful Schools study contributes to our existing knowledge base by 
confirming some relationshlos between positive school conditions and student 
achievement. However, what may work for successful, high poverty schools may not 
work for all high poverty schools. This preliminary study suggests that educators 
need to continue exploring methodologies and characteristics of successful schools 
that can be identified and transferred to other high poverty schools. 
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Overview 



What is the Florida Successful Schools 
project? 

The Successful Schools project is a research and 
development effort aimed at: 

• Identifying characteristics that distinguish 
successful, high poverty^ elementary schools; 

• Sharing research findings that will support school 
improvement, especially in areas that have large 
numbers of children from low-income families; 
and 

• Developing training and technical assistance 
strategies for schools that are operating (or plan to 
operate) Chapter 1 Schoolwide projects. 



How does the Successful Schools project fit 
in with Blueprint 2000? 

Part of the vision of Blueprint 2000 is that all 
schools will enable their students to achieve at the 
highest levels in a safe and orderly environment. 
The Successful Schools project is specifically 
designed for those schools that face the greatest 
challenge in realizing the vision of Blueprint 2000 
and whose success is absolutely critical to statewide 
reform in Florida. 

In support of Bluq)rint 2000, the Successful Schools 
project seeks to: 

• Better utilize the vast information resources of the 
Florida education data base for research, needs 
assessment, planning and evaluation to assist 
school improvement activities and 



Facilitate technical assistance linkages between 
high achieving, high poverty schools and similar 
schools that want (or need) extra help in raising 
the achievement level of their students. 



Who is involved in the Successful Schools 
project? 

The Successful Schools project is a collaborative 
effort,, consisting mainly of district Chapter 1 
evaluators and program, managers. Department of 
Educjition staff, and resource personnel from the 
Technical Assistance Centers in Atlanta, Georgia. 
A.t the present time there are about 40 people, 
representing over 20 Florida school districts, who 
are actively engaged in the research and 
development aspect of this project. They have 
contributed their time and expertise to get the project 
stalled. Expenses have been covered so far by 
existing budgets, mainly from Chapter 1 sources. 
Most of the 40 individuals involved in the Successful 
Schools project are part of an on-going Evaluation 
Advisory Panel for Chapter 1. 




'"Successful, high poverty schools" in this rqwrt refers to 
^looli that exceed state avenges oo a majority of reading, 
and writing assessmeots and alao have 65% or more of 
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students receiving Free or Reduced Priced meals. 
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How far along is tite Successful Schools 
project? 

This report covers the initial pilot phase of the 
Successful Schools project, which began in 
September, 1993. The specific objectives of the 
pilot included: 

• Analyzing data from the 1992-93 Florida School 
Reports and related sources to understand the 
impact of poverty on elementary schools, as well 
as conditions associated with higher achievement 
in schools that serve large numbers of children 
from low-income families; 




• Adapting and field testing staff and paient survey 
instruments that were based on previous, 
"effective" schools research; and 




• Visiting a sample of high poverty schools to 
observe and interview staff to enrich self-reported 
paper and pencil survey data. 




What are the results of the Successful 
Schools pilot project? 

The data base analysis verifies what most people 
already know or suspect-that schools with high 
concentrations of students from low-income families 
^ctniggie against nearly overwhelming odds. It also 
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shows what many people do not know or recognize- 
that some high poverty schools actually do better 
than most schools, despite the odds. 

The data base analysis looks first at the larger 
context of all 1,458 Florida elementary schools in 
1992-93 for the purpose of examining the impact of 
poverty on student achievement, school learning 
environment, student characteristics, and school 
inputs in general. The results show that despite 
comparable resources, student achievement in high 
poverty schools lags far behind what is t;. iiical for 
low and medium poverty schools. 

The focus then narrows to the 368 elementary 
schools that have the highest concentrations of 
children from low income families. Here, the 
purpose is to examine conditions that are related to 
higher achievement among high poverty schools. 
There is also a special section that features the 29 
highest achieving schools in contrast with an equal 
number of the lowest achieving schools-all of which 
were high poverty. A major conclusion is that the 
Florida School Reports and the data bases upon 
which they are built can offer clues but cannot 
definitively point to solutions for raising student 
achievement. Areas that warrant further 
consideration are the role of support staff and better 
utilization of federal funds. 

Results from the survey and on-site observation 
phases narrow the research focus down to the 16 
schools that participated in the field-test of 
instruments and procedures. The purpose here was 
to determine the feasibility of using information in 
addition to the state data base to expand our 
knowledge of what works in Florida's high poverty 
schools. 

Preliminary results from the staff surveys indicate 
that a valid, reliable, and much shorter instrument 
can be extracted from the original pool of 1 17 items, 
covering the seven correlates of "effective" schools 
(see page 16). Items related to Safe and Orderly 
climate revealed the greatest differRncc between high 
and low achieving schools. Instructional leadership 
and High Expectations for student achievement also 
showed differences. 
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Parent surveys were also found to be reliable; 
however, the responses in general, did not clearly 
distinguish between high and low achieving schools. 
Rather, parent responses appear to be more related 
to learning environment. The results also reveal 
how often parents say they don't know about such 
important aspects of school reform as the school's 
purpose, sharing ideas for school improvement, or 
the activities of parent organizations. 

The procedural aspects of on-site visits were 
analyzed and recommendations made for 
refinements. All visits were carried out by teams of 
four staff members, none of whom were from the 
district where the school was located. Under the 
leadership of a district person, each team spent one 
full day interviewing school personnel and casually 
observing the site. 

Results from team debriefings at the end of each 
visit revealed that: teams describe discipline and 
leadership in a positive way more often in high 
achieving schools; stal^ development is not referred 
to in low achieving schools; but change is mentioned 
four times more frequently in low than in high 
achieving schools. 

During the course of on-site visits, over 200 school 
personnel were interviewed and their comments 
recorded. Over 250 pages of transcribed notes were 
taken for later analysis using qualitative methods. 
Results indicate that overall responses to on-site 
interviews were not notably different between high 
and low achieving schools. There were, however, 
subtle differences detected with regard to 
Instructional Leadership, School Mission, and Safe 
and Orderly Environment. 

As a result of the pilot project phase, the purpose 
and activities associated with on-site observations are 
being reconsidered. Perhaps it would be more 
profitable to follow up on such school conditions as 
the role and preparation of support staff, inservice 
training, resource utilization and student discipline. 
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Why look at school poverty? 

From a national perspective, poverty remains a 
roadblock to educationsd excellence, despite years of 
concerted efforts to alleviate the effects of poverty 
and renewed efforts at school reform. A generation 
after the great "War on Poverty" was declared, the 
National Assessment of Chapter 1 (1992) indicated 
that conditions have not improved: 

As poverty among the elderly fell in the 1980' s, 
children became the poorest of all Americans. In 
1991, more than 13 million children in the United 
States-more than one in five-live in families 
with income below the poverty line (Committee 
on Ways and Means, 1992). In 1990, 8.6 million 
school-age children (ages 5-17) were living in 
poverty (U.S. Department of Labor, 1990). 

Recent increases in the number of poor children in 
Florida were not as high as in many other states, but 
the overall poverty level remains slightly higher than 
the nationsd average. According to Hodgkinson 
(1993): 

The number of children in Florida who are poor 
increased from 18.5 percent to 19.9 percent 
during the 1980's, while nationwide this figure 
increased from 16.0 percent to 19.5 percent. 

Coming from a low income background is one thing; 
attending a high poverty school is quite another. 
Research studies point out that high concentrations 
of poor students in schools, not the poverty status of 
individual students, is the major problem. 

How was school poverty determined? 

Though far from perfect, one of the most common 
ways of looking at school poverty is in terms of the 
percent of students who receive Free or Reduced 
Priced Lunch. In 1992-93, the percent of 
elementary students receiving federally subsidized 



meals in Florida schools ranged from zero to 99% 
with 46.7% being the median. 



What data were used and what were the 
sources? 



A benchmark study of school poverty (Kennedy, 
Jung, and Orland, 1986) divided a sample of the 
nation's schools into low, medium, and high poverty 
groups according to the percent of students on Free 
or Reduced Priced Lunch relative to the state 
average. The same procedure was used for Florida's 
elementary schools: 

• Low poverty = schools with poverty 
concentrations in the lowest quartile in the state. 
This included the 353 schools that had zero to 
29.2% of their students on Free or Reduced Priced 
Lunch; 

• Medium poverty = schools with poverty 
concentrations in the middle two quartiles. There 
were 737 such schools and the percent of students on 
Free or Reduced Priced Lunch ranged from 29.3% 
to 64.7%. 

• High poverty = schools with poverty 
concentrations in the upper quartile in the state. 
Some 368 schools were included in this group and 
the percent of students on Free or Reduced Priced 
Lunch ranged from 64.8% to 98.9%. 



School Poverty Classification 

by Quartile Group 



Almost all of the data came directly from the 1992- 
93 Florida School Reports (Technical Version) that 
were disseminated to all elementary, middle and 
high schools in November, 1993. These reports 
represent school level aggregates from the Florida 
Department of Education's Stiident and Staff Data 
Bases. A total of 20 variables were examined for 
evidence of significant differences betv/een low, 
medium and high poverty schools. These 20 
variables were organized into four groups. 

1. Student Achievement is Goal 3 under Blueprint 
2000 and is prominent in the public perception of the 
quality of education. Though the choice of 
achievement measures continues to be hotly debated 
among the major stakeholders in educational reform, 
the 1992-93 Florida School Reports offered three- 
reading, math and writing-all of which were for 
fourth grade students. 

2. Learning Environment addresses Blueprint 
2000' s Goals 4 and 5. Variables in this group 
reflect on the progress of students' education, 
opportunity to learn, and a safe and orderly school 
environment. Data on the following variables were 
for all students, not just fourth graders: promotion, 
attendance, and out-of-school suspension. 

3. Student Characteristics are important to consider 
when identifying model programs or linking 
technical assistance providers. Schools that want or 
need extra help in their improvement efforts will be 
more receptive if the sources of model programs or 
technical assistance share similar features, such as: 
percent of minority students, percent of Limited 
English Proficient (LEP) students, and percent of 
students who change or re-enter a school (mobility 
rate). 





M.9% 
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4. School Inputs are the physical, financial and 
staff resources that schools have to accomplish the 
job of educating Florida's youth, including: school 
size, class size, support staff, "Regular" program 
expenditures (consisting of per pupil expenditures 

3 



from general fiind sources, plus per pupil 
expenditures firom federal sources), year 
teachers, experienced teachers (those with 10 or 
more years in the profession), teachers with 
advanced degrees, average teacher salaries, and 
percent minority teachers. 



What Umitations apply to the data? 

Unlike others, this study of the impact of poverty is 
not subject to any appreciable amount of sampling 
error, because all elementary schools in the state 
during 1992-93 were included. However, there are 
other limitations that should be considered. Despite 
extensive efforts to verify School Report data, 
inaccuracies may exist due to varying degrees of 
quality control at the state, district, and school 
levels. Also, not all variables were reported for all 
schools. Another set of limitations has to do with 
data definitions, calculation methods and the fact 
that some detail is lost when data are collapsed for 
reporting purposes. Further, conclusions are limited 
to elementary schools only. 



considered statistically significant^ the analysis went 
on to estimate practical significance for program 
managers, principals, teachers or anyone else trying 
to understand and reduce the impact of poverty on 
schools. The practical significance of relationships 
between poverty and school report variables was 
determined by calculating effect sizes which were in 
turn classified as small, medium or large in 
accordance with procedures described by Cohen 
(1988). 



What were the results? 




What methods were used for analysis? 

Most of the data in the Florida School Reports are 
percentages or rates (e.g., the percent of students 
scoring above the national median in reading or the 
school's attendance rate). Other data were converted 
to percentages for analysis purposes. To cover all 
20 variables, the data were considered to be 
generally ordinal in nature. This means that figures 
for one school can be compared to another (ranked) 
in terms of being higher or lower, but the 
differences are not uniform. For this type of data, 
median values are probably the most appropriate for 
what is "average" or typical for all schools, as well 
as for groups of similar schools. 



All 20 data base variables showed evidence of a 
significant relationship with increasing levels of 
school poverty. For 13 of the variables, the 
influence of poverty was shown to have considerable 
practical significance, as well. Effect sizes for five 
variables were classified as large, four were 
medium, and four were small. In the following 
discussion, symbols are used to help distinguish 
among variables with different effect sizes : ♦ for 
large, ❖ for medium, for small, and D for 
marginal, but statistically significant. 



The impact of poverty on each variable was 
determined by analyzing the number of low, medium 
and high poverty schools that were above and below 
the overall state median. When results were 
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^Probability of chance occurrence \esi than one in 100 
(p<.Ol). 
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Student Achievement assessment fonnat. In the Florida Writes', prognim, 

trained raters judged tl^ie merits of student 
As shown below, analysis of the impact of school compositions against rubrics or standards with scores 
poverty on student achievement shows a consistently ranging from zero' to six. The percent of students 
large, negative relationship as the concentration of in high poverty schools who scored at or above the 
students from low-income families increases. midpoint (three* or better) was about one-half that of 

medium poverty and about one-third 
Dinnipil©fi ©If P©W©D% ©OrO of ^t^^^nts in low poverty 

Student Achievement 



% of students Above Standard* 



Poverty UvM 




Reading 



Math 



Writing 



Low 


64% 


70% 


28% 


Medium 


49% 


56% 


19% 


High 


25% 


37% 


9% 



'Reading tk Math: National Madlan %tl« 
Writing: 3 or alMva on Flort4a Wrltttl 



♦ Reading achievement in high poverty schools 
tends to be far below the overall state median and 
even further below the performance level of students 
in low poverty schools. The percent of fourth 
graders scoring above the national median in high 
poverty schools was about two and a half times less 
than in low poverty schools. 

4 Math achievement was also dependent on the 
poverty level in schools. Students in high poverty 
schools tended to score lower than the median for 
the state and much lower than students in low 
poverty schools. The percent of fourth graders 
scoring above the national median in high poverty 
schools was about half that of low poverty schools. 

4 Unlike reauing and math achievement that was 
measured on norm-referenced tests, writing 
proficiency was determined through an alternative 
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In summary, when uniform measures 
are used to gauge student 
achievement, it is clear that the lowest 
performing schools will most likely be 
those with the highest concentrations 
of students from low income families. 
Without careful attention to these 
schools, the influence of poverty may 
overwhelm the good intentions of 
school reform. 

On the other hand, there were notable 
exceptions to the general trend of very 
low achievement among high poverty 
schools: 



Sixty high poverty schools (or about 16%) had 
reading scores that were above the statewide 
median for all schools. Even more encouraging, 
32 high poverty schools (9%) had reading scores 
above the median for low poverty schools. 

There were 75 high poverty schools (20%) that 
had aggregate math scores at or above the 
statewide median; 34 (or 9%) had scores at or 
above the median for low poverty schools. 



'a zero was assigned to compositions that were luscorable due 
to students' &ilure to respond, unintelligible responses, or 
being completely off the assigned topic. 

paper scored 3 generally focuses on the topic but may 
contain extraneous or loosely related information, exhibits 
some evidence of an organizational pattern, has little 
development of tnippoiting ideas, and may contain some errors 
in spelling and punctuation conventioiu. 
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There were 41 high poverty schools (11%) in 
which writing performance was at or above the 
state median. Though rare, there were 6 high 
poverty schools (2%) whose writing scores were 
above the median for low poverty schools. 



Learning Environment 

The data below show relationships between poverty 
and selected indicators of a school's learning 
environment. 

Promotion and Attendance 




Promotion Rat* Attondanc* Rat* 



Low Q 


97% 


9S% 


Medium □ 


96% 


94% 


High ■ 


91% 


94% 



•> As indicated by a medium effect size, a school's 
promotion rate is somewhat dependent on its poverty 
level. The median promotion rate in high poverty 
schools was about five percentage points less than in 
medium poverty schools and about six percentage 
points less than in low poverty schools. These 
results take on even more importance given the fact 
that fully 81% of all dropouts in 1992-93 were one 
or more years over age, meaning they had likely 
been retained at least once. 

*> Despite extremely small differences in attendance 
rates' among low, medium and high poverty 
schools, the effect of increasing poverty level was 
sufficiently consistent to warrant a medium 
classification. 



'The relative lack of differences in attendance rates may be a 
function of how they were calculated in the 1992-93 Florida 
School Reports. Attendance rates reflected the number of days 
preaeot divided by the number of days present + number of 
days absent, regardless of the length of school enrollment. 
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Suspension Rate 



P*ro*nt of SttudMiU 




In comparison to promotion and attendance, out- 
of-school suspension rates were relatively low and 
are displayed on a different scale so as not to 
diminish tilieir importance. The data above show 
that out-of-school suspensions are at least partially 
dependent on school poverty level. In high poverty 
schools the median suspension rate was over three 
times higher than in low poverty schools. 




While not as dramatic as student achievement, 
variables that reflect a school's learning environment 
do have a bearing on the quality of education 
available to students in Florida. Schools which 
serve a high proportion of poor students have lower 
promotion and higher out-of-school suspension rates. 
Attendance rates were also lower in high poverty 
schools, b'^t differences were minimal. 

Among high poverty schools there were notable 
exceptions to the general trend: 

Seventy-two high poverty schools (20%) had 
promotion rates equal to or higher than the 
statewide median and 34 (or 9%) had promotion 
rates that were the same or better than in low 
poverty schools. 

Many high poverty schools had relatively low 
suspension rates: 138 (or about 38%) had rates 
equal to or lower than the state median, and 96 




(or 26%) had rates equal to or lower than the medium poverty schools and 19 percentage points 
median for low poverty schools. higher than for low poverty schools. 



Student Characteristics 

Student characteristics which are unique to high 
poverty schools need to be considered for at least 
two reasons. First, it is important to understand the 
school clientele who are affected in high poverty 
schools. Second, such knowledge can facilitate the 
establishment of viable technical assistance linkages 
between successful and lower achieving schools. 

student Characteristics 

P*re«nl ef ttudwitt 




Poverty L«Ml 




Minority 



School MoMMy 



Low 


21% 


25% 


Medium 


28% 


35% 


High 


76% 


44% 



♦ The data above reveal a very strong relationship 
between the proportion of minority* students in a 
school and its poverty level. While the percent of 
minority students across the state converts to about 
one in three students (31%), that ratio jumps to over 
three out of four in high poverty schools. This is in 
stark contrast to about one in five (21%) minority 
students in low poverty schools. 



❖ 



The data above also indicate that student 
mobility' is somewhat dependent on school poverty 
level. The median mobility rate for high poverty 
schools was nine percentage points higher than for 



'Minority students include Black, Hispanic, Asian or Pacific 
Islander, and American Indian. 

'^Student mobility rates in the 1992*93 Florida School Reports 
were derived by comparing enrollment at the beginning of the 
school year widi the combined total of enrollments, reentries 
and withdrawals occurring during the school year. 
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0 Though statistically significant, the relationship 
between the percent of Limited English Proficient 
(LEP) students in a school and its poverty level is 
weak. Regional differences probably account for 
more difference among schools than does poverty, 
hence its negligible effect size. 



Analysis of relationships between school poverty 
levels and selected student characteristics yields 
mixed results. On the one hand, there is a very 
strong relationship between high poverty and high 
minority composition in schools. The relationship 
between poverty and student mobility was 
considerable, as well. On the other hand, a 
negligible effect size was found for the percent of 
Limited English Proficient students in low, medium 
and high poverty schools. 

School Inputs 

School inputs reflect the adequacy of physical, 
human and financial resources which schools have at 
their disposal to meet the high standards for student 
learning which are expected under Blueprint 2000. 

D One of the most obvious features about schools is 
the number of students they serve. While high 
poverty schools actually tend to be smaller, the 
practical significance of this finding was minimal. 

D Probably one of the hotter discussion topics 
among educators is tlie importance of smaller, more 
manageable class sizes'; however, only minimal 



'In the 1992-93 Florida School Rqmrts, class-size was 
determined by dividing the unduplicated count of students 
enrolled in specified courses by Uie unduplicated count of 
teachers in those same courses. Also, the following disclaimer 
appears in the Technical Version. 'The avenge class-size may 
or may not reflect the actual number of students in a given 
class. However, this does represent an average number of 
students within the subject areas or grades indicated.' 
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differences favoring high, instead of low or medium 
poverty schools were found. 

# of Support Staff 




Low E 



12 
15 
19 



In addition to teachers and administrators, an 
important ingredient of a school's human resources 
can be found in its support staff. Beyond a standard 
complement of clerical, food service, transportation 
and custodial staff, there can be teachers on special 
assignment, administrative assistants, health service 
providers, teacher aides, tutors, computer 
specialists, resource officers-and many others. The 
data above show a distinct advantage for high 
poverty schools in the number of support staff per 
500 students. On average, high poverty schools had 
four more support staff than medium poverty 
schools and ;seven more than low poverty schools. 
Unfortunately, there was no way to differentiate 
types of support staff in the Florida School Reports. 
This is an area that warrants follow-up research in 
terms of the different kinds of support staff 
employed in high poverty schools and the extent and 
nature of their training. 

Student Expenditures 

Types and amounts of fmancial resources are among 
the most important school inputs that can be readily 
identified. Florida School Reports contain several 
indicators of resource availability with "Regular 
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Program" expenditures' being perhaps the most 
relevant. Regular program expenditures include two 
major components: general funds (local and state 
general revenue) and federal funds (primarily 
Chapter 1 for elementary schools). 

Unfortunately, expenditure data tend to be 
confounded by differences between school districts, 
primarily due to variation in local tax base and cost 
of living. In order to isolate the impact of poveity, 
the data base analysis looked at the percent more or 
less that schools spent in relation to their own 
district's average expenditures. 

p>®w©i% §L S©lhi®®D linipyiJ© 
Expenditure Differences 
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The data above suggest that "regular" 
expenditures contained in the Florida School Reports 
are at least somewhat related to the poverty level of 
schools, but not in the direction one might initially 
expect. High poverty schools spent about 8% more 
per pupil than their district average and about 13% 
more Uian low poverty schools. Thus, contrary to 
popular notion, insufficient resources do not 
necessarily characterize schools that serve large 
numbers of students from low income families. 
However, the whole story is not revealed by looking 
simply at "regular program" expenditures; the 
components need to be examined, as well. 

□ Analysis of the general fund component of regular 
program expenditures indicated a statistically 



'Expenditures include: salaries + benefits + purchased 
services + materials and supplies + other direct expenses + 
capital outlay + school indirect cost + district indirect cost. 
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significant, but minimal relationship with school 
poverty. Again the data reflected that high poverty 
schools actually spent about the same as medium and 
low poverty schools. 

♦ The graph on the previous page reveals relatively 
large differences in federal fiind expenditures that 
favor high poverty schools by about one and one- 
half times their district average and about six times 
that of medium poverty schools. Low poverty 
schools spent about 80% less than their district 
average. This finding would suggest that Chapter 1 
funds in Florida are generally concentrated where 
there are large numbers of children from low income 
families, as intended by federal legislation. The fact 
that a large effect size is associated with this variable 
is even niors persuasive. 

Teacher Characteristics 
Teacher Experience/Characteristics 
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The proportion of first year teachers in a school 
reflects tfie lowest level of instructional experience 
among a school's certified staff. The data above 
show a significant, but relatively small relationship 
between the percent of first year teachers and school 
poverty level. However, closer examination reveals 
that the median percent of first year teachers in high 
poverty schools was about double what was found in 
both low and medium poverty schools. 

D The proportion of teachers in a school who have 
at least ten years' experience is another variable that 
can influence the quality of instruction available in a 
school. However, a statistically significant, but only 
minimal relationship between teacher experience and 

o 
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school poverty level was found. As expected, high 
poverty schools are in fact staffed by proportionately 
fewer experienced teachers, primarily because they 
have more first year teachers. 

D The proportion of teachers with advanced degrees 
is indicative of the relative level of training that 
teachers have in a particular school. While the 
differences among high, medium, and low poverty 
schools were statistically significant, the effect size 
was marginal for this variable. 

0 Differences in teacher salaries among low, 
medium and high poverty schools can also be viewed 
as an indirect reflection of how resources are 
expended to provide the quality of instruction 
necessary for attainment of high student 
performance. However, when analyzed as a percent 
of average salaries within a district, differences 
between low, medium and high poverty schools 
were minimal. 

❖ What about the characteristics of instructional 
staff? The data in the chart to the left show that the 
proportion of minority teachers in a school is 
somewhat dependent on its poverty level. It can be 
seen that the percent of minority teachers in high 
poverty schools was much higher than the state 
average and nearly double the figure for low poverty 
schools. As well, this variable had the second 
largest effect size of all twelve input variables 
examined. This finding is related to the fact that 
high poverty schools also tend to have large numbers 
of minority students. 

What conclusions can be drawn about the 
impact of poverty on elementary schools? 

Clearly, increasing levels of poverty extract the 
greatest toll on student achievement. In this study, 
all three indicators of aggregate school achievement 
(reading, math, and writing) showed evidence of a 
large, negative relationship as poverty levels 
increased. 

To a lesser degree, school poverty also has a 
debilitating effect on a school's learning 
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environment-promotion rates in high poverty 
schools are lower and suspension rates are higher as 
compared to schools with moderate or low numbers 
of poor children in membership. 

Schools that have high concentrations of students 
from low income families tend to have relatively 
high proportions of minority students. Their 
students also tend to change schools more frequently 
than others. 

Contrary to what might be expected, high poverty 
schools actually have more money to spend on 
students than lower poverty schools in the same 
district. There is a slight advantage in favor of high 
poverty schools with regard to pupil expenditures 
from general fund sources and a huge advantage in 
federal funds. 

While high poverty schools tend to have more 
support service personnel to help in the task of 
educating disadvantaged students, they also tend to 
have more inexperienced, first year teachers and 
correspondingly fewer experienced teachers. Just as 
high poverty schools tend to have more minority 
students, they also have more minority teachers. 




Lest this summary of fmdings paint too dismal a 
portrait of high poverty schools, it needs to be re- 
emphasized that numerous exceptions were found: 
16% of Jiigh poverty schools had reading scores 
higher than state averages; 20% had higher math 
scores; 11% had better writing scores; 20% had 
better promotion rates and 38% had lower 
suspension rates. These fmdings suggest that 
roughly one fifth of all high poverty schools actually 
did quite well despite the great challenges they 
faced. 



How is this part of the report different? 

The purpose of this section of the report is to narrow 
the focus of research to those schools that are of 
greatest interest to Chapter 1 program managers and 
which probably represent the greatest challenge in 
realizing the goals of Blueprint 2000. Instead of 
looking at all 1,458 elementary schools in Florida, 
the research context was limited to the 368 schools 
classified as high poverty. Also, the study emphasis 
was shifted from the general impact of poverty to 
specific conditions that are related to student 
achievement. 

Were different methods used? 

The statistical methods were the same; however, the 
data were categorized in a different way. Since 
raising achievement was of greatest interest, reading, 
math and writing achievement for each of the 368 
schools were categorized as being either above or 
below the median for all high poverty schools. 
Student characteristics and school inputs were also 
categorized as being above or below the median for 
high poverty schools'*'. The objective was to find 
conditions associated with higher achievement levels 
in hopes they would point to specific areas for 
improvement efforts. 

As results are presented, the following symbols are 
used to designate large ♦ , medium ❖ , small , 
and marginal, but statistically significant effect 
sizes D. 



Clearly, these exceptions suggest something good is 
happening in some high poverty schools. But what? 
Qui the school reports or related data point the way 
to improved achievement? 
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'^^Vhile raising the achievement level of high poverty schools 
above their group median may not represent a standard of 
excellence, it does represent a standard of progress. The 
objective was to find conditions associated with higher 
achievement in hopes of targeting specific areas for 
improvement efforts. 



What were the results for reading? 

The results from this part of the study indicate that 
the percent of schools whose students scored above 
the median reading level for all high poverty schools 
increased H when: 

♦ The percent of minority students was lower 

♦ The percent of students on Free or Reduced 
Priced Lunch was lower 

❖ The percent of minority teachers in the school 
was lower 

❖ Differences between school and district 
expenditures of federal (Chapter 1) funds were 
smaller 0 

The number of support staff (per 500 students) 
was higher H 
-❖■The percent of Limited English Proficient 
students was lower ^ 

Relationships between reading achievement and the 
following conditions were also statistically 
significant; however, their effect sizes were 
marginal: promotion rate, percent of first year 
teachers, and number of students enrolled in the 
school. 

It is also important to note that the following 
variables were not significantly related to reading 
achievement in high poverty schools: attendance 
rate, suspension rate, mobility, class size, "Regular" 
program expenditures and its general fund 
component, percent of experienced teachers in the 
school, percent of teachers with advanced degrees or 
average teacher salaries at the school. 

What were the results for math 
achievement? 

The results suggested that math achievement in high 
poverty schools increased H when: 

❖ Differences between school and district 
expenditures of federal funds were smaller -0 

■❖■The percent of students on Free or Reduced 
Priced Lunch was lower 0 

o 
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"❖■ The percent of minority students in the school 
was lower -0 ' 

The relationships between math achievement and the 
following conditions were statistically significant; 
however, their effect sizes were marginal: percent 
of minority teachers, differences in "Regular" 
program expenditures, and suspension rate. No 
other database variables were significantly related to 
math achievement. 

What were the results for writing 
performance? 

There were only two conditions that showed 
evidence of even a small effect. Writing 
performance increased H when: 

-❖■The percent of students on Free or Reduced 

Priced Lunch decreased -0 
■❖■ Differences in federal expenditures were 

smaller 

There were a few other variables which exhibited 
statistically significant, but very small effect sizes: 
average teacher salaries and the percent of minority 
students. No other data base variables exhibited 
significant relationships with writing achievement in 
high poverty schools. 

Were there conditions that seem to effect 
learning environment in high poverty 
schools? 

There were no variables examined that showed even 
a small effect on promotion, attendance or 
suspension rates among high poverty schools. 

What conclusions can be drawn? 

Looking at the relationship of data base variables 
across achievement measures, th<3 percent of students 
on Free or Reduced Priced Lunch has one of the 
strongest set of relationships. This finding is 
particularly interesting since all of the schools in the 
analysis were already classified as high poverty. Yet 
when the percent of students on Free or Reduced 
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Priced Lunch was above the median (79%), 
achievement was lower. This suggests that the effect 
of increasing poverty concentration does not level 
out; it remains a potent influence even in "high 
poverty" schools. While program managers have 
little control over school poverty, it is something 
that should be brought to the attention of policy 
makers in charge of school zoning and transfers. 

Federal expenditures is another variable that stands 
out from tihe list of conditions that are related to 
scliool achievement. This fector had a moderately 
strong relationship with reading and math and a 
smaller one with writing. However, the direction of 
the relationship is opposite of what one might wish. 
As the difference between a school's expenditure of 
federal funds versus the district average increases, 
reading, math and -A^riting achievement appear to 
decrease. These data have implications for 
Chapter 1 Schoolwide projects, where extra 
resources are concentrated in high poverty schools. 
The pattern of negative returns argues that money 
alone cannot overcome obstacles to improved 
achievement associated with large numbers of 
students from low-income families. Further 
questions need to be addressed: How are 
Schoolwide Project funds coordinated with school 
improvement plans as required under Blueprint 
2000? To what extent are these projects based on 
the specific academic needs of students? 

The percent of minority students was strongly 
related to reading, but had a much smaller 
relationship with math. Also, the percent of 
minority teachers in a school exhibited a moderate 
relationship with reading. As previously noted, the 
percent of minority teachers tends to be directly 
related to the percent of minority students in 
schools. 

The number of support staff per 500 students 
appears to have a positive, though small, influence 
on reading achievement. As mentioned before, this 
is a variable that warrants extended investigation, 
because it is not clear what kinds of support staff are 
involved in promoting achievement or how they are 
utilized. 



From a different perspective, it is informative to 
examine which conditions are not related to 
achievement outcomes. Surprisingly, mobility was 
not signiflcantly related to any of the three 
achievement measures, though it is often cited as a 
reason for poor achievement. It is also interesting 
that the general fund component of "Regular" 
program expenditures did not show evidence of a 
significant relationship with any of the three 
achievement measures. Just the opposite was true 
for federal expenditures. 



ACAlNSTAtLODDSj SUCCESSFUL, HliGH 




This part of the report 
represents yet another 
narrowing of research 
perspective, down 
from all high poverty 
elementary schools 
(n=368) to a smaller 
subset of those same 
schools (n=58) that 



represents the extreme ends of an achievement 
continuum~the highest and lowest achieving. The 
purpose here is to capitalize on a spin-off of the 
research previously reported-that some high poverty 
schools actually did quite well, despite 
overwhelming odds. 

What methods were used to identify and 
study "successful", high poverty schools? 

High poverty schools which exceeded overall state 
medians in at least two out of three achievement 
areas (reading, math, or writing) were identified as 
successful. Included in this group were 29 schools, 
located in 20 of Florida's 67 districts, whose 
students did better than most other schools in the 
state, even though their high poverty status would 
predict the reverse. This was the "successful" 
schools group. 
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By way of contrast, lower achieving schools were 
defined as ones which fell below the median for high 
poverty schools on at least two out of three subject 
areas. On this basis, there was a total of 160 
schools (43%) that needed extra assistance. In order 
to have equal size groups and to present the sharpest 
contrast against "successful" schools, 29 of the 
lowest achieving were subsequently identified. This 
was the "comparison" schools group. 

Together, the two subgroups consisted of 58 out of 
368 (or 16%) of those schools which had the highest 
concentrations of students from low income families. 
Combined, the median percent of students on Free 
or Reduced Priced Lunch was 80%, which is above 
the eligibility cutoff (75%) for Chapter 1 
Schoolwide projects. Incidentally, seventeen of the 
58 schools (35%) actually were operating 
Schoolwide projects at the time of the study; of 
these, only three were in the successful school 
group. There were fourteen Schoolwide Projects in 
the comparison group. 

As results are presented, differences between 
successful and comparison schools are shown 
relative to medians for all elementary schools in the 
state (n= 1 ,458). While exceeding the state averages 
may not represent the very highest of goals 
envisioned by Blueprint 2000, it is a remarkable 
accomplishment for high poverty schools. 

What were the differences between 
"successful" and "comparison" schools? 
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The preceding figure provides graphic evidence that 
the 29 successful and 29 comparison schools were 
worlds apart with regard to student achievement. 
Students in the successful schools group scored 
higher than statewide medians in all subject areas- 
reading, math, and writing. They also scored 
dramatically higher than students in comparison 
schools. 

The figure below shows that differences on variables 
that reflect learning environment were not as 
pronounced as those that deal directly with 
achievement. In general, promotion and attendance 
rates in successful, high poverty schools were lower 
than overall state figures and higher than comparison 
schools. 
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Using a different scale, the figure below shows that 
successful schools had a much lower out-of-school 
suspension rate than comparison schools. 
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As shown below, high and low achieving schools 
varied somewhat on one key variable and greatly on 
another. 
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Successful schools averaged about twelve percentage 
points lower than comparison schools on the number 
of students from low income families, but still about 
20 percentage points more than the state median. 
Successful high poverty schools had more minority 
students than the statewide median, but fewer than 
their lower achieving counterparts. There was 
virtually no difference with regard to percent of LEP 
students or mobility rates. 

The following figures show information about school 
inputs in successful and lower achieving schools. 

School Size 
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Successful schools tended to be about 100 students 
smaller than the state median and almost 200 
smaller than the median for low achieving schools. 



Staff Ratios 
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Class-sizes in high achieving schools were slightly 
smaller than in the comparison schools, which had 
about the same as the statewide average. There was 
a big difference in relative numbers of support staff 
that favored high versus low achieving schools- 
about six staff members more per 500 students. 



Expenditure Differences 
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The figure above shows the percent more or less that 
schools spent than their respective district averages. 
There was very little apparent difference in "regular" 
fund expenditures between successful and 
comparison schools, both spent a little more than 
their districts' average. However, when fund 
sources were broken out, it was found that 
successful schools spent about 10% more per pupil 
in general funds, but about one and half times less in 
federal funds. 
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Teacher Experience/Characteristics 



Percant of Teacher* 



100% 




program expenditures, or teachers >vith advanced 
degrees. 

Successful schools were clearly distinguished from 
their lower achieving comparison group by having 
fewer minority students/fewer minority teachers, 
more support staff and lower suspension rates. 
Successful schools were somewhat distinguished 
from comparison schools by smaller numbers of 
students on Free or Reduced Priced Lunch, smaller 
total number of students, more general fuiads/fewer 
federal funds, fewer first year teachers/more 
experienced teachers, and slightly higher teacher 
salaries. 



The preceding graph looks at differences in teacher 
experience and training: the percent of first year 
teachers in both groups was higher than the state 
average, but was most pronounced (almost double) 
among comparison schools. Successful schools had 
a larger percentage of experienced teachers than 
comparison schools, but both groups were 
considerably below the state median for all 
elementary schools. Similarly, the proportion of 
teachers with advanced degrees was a little higher in 
successful schools, even higher than the state 
median. Successful schools had about the same 
proportion of minority teachers as the state median, 
but less than one-third the proportion found in 
comparison schools. 

What conclusions can be drawn from these 
comparisons of "successful" versus lower 
achieving schools? 

First, there is compelling evidence in the Florida 
School Reports that some high poverty schools do 
extremely well at promoting high levels of student 
achievement. On the other hand there is a relatively 
large number of high poverty schools that are not 
doing well and some that are in dire need of 
improvement. However, successful and comparison 
school groups were not distinguishable on such 
measures as promotion and attendance rates, percent 
of LEP students, mobility, class size, "regular" 



STAFF AND PARENT SURVEYS 



^ So far, the emphasis has been 
on information available 
i^^f through the Florida School 
AHB Reports, which has been shown 
«^V« to reflect conditions that exist in 
■ ■ I schools, but not where to target 
improvement efforts. The next phase of research 
explored staff and parent surveys as an alternative 
source of information. 

How were the survey instruments 
developed? 

The research team turned first to current literature 
on "effective" schools and located existing 
instruments that had been used in San Diego". A 
great deal of effort was spent reviewing and adapting 
an instrument that would be suitable for Florida 
elementary schools, especially those serving large 
numbers of poor students. It was decided that the 
survey should ask about school conditions that can 
be tied directly to seven conelates of "effective" 
schools, as follows: 

Clear School Mission : There is a clearly stated 
mission that focuses on academics. 
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Frequent Monitoring : There are regular 
assessments of student progress and the 
effectiveness of school programs. 

Safe and Orderly Learning Environment : The 
school is safe and the students conduct themselves 
in an orderly manner with all parties engaged in 
purposeful activities that are learning related. 

High Expectations : The school operates on the 
belief that all students can learn the basic and 
higher order skills that are a part of the school 
curriculum. 

O pportunity to Learn and Time on Task : The 
school regards learning time as a critical resource 
and manages it well through classroom procedures 
and interactive learning. 

Instructional Leadership : Principals demonstrate 
strong leadership, especially in the areas of 
curriculum and instruction, and they are able to 
^are leadership by involving other staff members 
in leadership activities and positions. 

Home-School-Community Relations : Parents and 
the community understand and actively support 
the purpose of the school and are treated as 
partners in their children's education. 

The field-test version of the staff survey had 117 
questions aimed at teachers, principals and 
paraprofessionals. It was known in advance that 
fewer questions would make the survey easier to use 
and better received. Therefore, one goal of the 
field-test was to reduce the number of survey 
questions and maintain reliability. 

A parent survey was also developed for field-testing. 
This was a 23 item survey that was based mainly on 
the Home-School-Community Relations correlate. 
Three versions of the survey were created—English, 
Spanish, and Haitian Creole~to accommodate 
cultural diversity. Most schools distributed the 
parent survey through a sample of students; some 
schools attempted a much broader distribution. 



How many schools were involved in the 
survey field-test? 

A total of 16 schools were involved in testing the 
survey instruments and on-site observation 
techniques. These 16 schools were located in six 
Florida school districts from the Panhandle to South 
Florida. Some of the pilot schools could be 
classified as urban, some rural, and others suburban. 




All pilot schools had at least 75% of their students 
on Free or Reduced Priced Lunch and were 
therefore eligible for extra federal resources through 
Chapter 1 Schoolwide projects, although only seven 
were operating such projects at the time of the study. 
In the majority of pilot schools, at least half of their 
students were minorities. Some of the schools 
served large numbers of Limited English Proficient 
(LEP) and/or migrant students. 

How were test sites selected? 

A key feature of the research design was to contrast 
high versus low achieving schools in order to 
identify unique characteristics of success. Since the 
study was being conducted by Chapter 1 evaluators, 
they used that program's data base to make a 
preliminary determination of candidate schools. At 
first, more schools were identified than were 
needed. About half had some of the best reading 
gains in the state over the last two years; about half 
had some of the lowest gains. 
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Next, the list of potential schools was examined for 
characteristics that would be most appropriate for the 
pilot (e.g., continuity of administration, variety of 
student membership, geographical representation, 
etc.). Finally, the list was narrowed to 16 schools 
that were in the advisory panel's own districts in 
hopes of facilitating communication and 
participation. The original target number of pilot 
schools was 8-10; however, when the invitation to 
participate was extended, superintendents in all six 
districts accepted and all 16 schools participated. 

guccessful Schools Staff Survey 
How were the surveys validated? 

A key objective of the field-test of both staff and 
parent surveys was to determine the reliability and 
validity of the instruments. This was important both 
for analyzing results from the 16 pilot schools and 
for future use with a much broader sample of high 
poverty schools. The reliability'^ o^ the two 
instruments was determined on the basis of the 
internal consistency of items or groups of items in 
terms of measuring the same concepts~in this case 
correlates of effective schools. Face validity of the 
instruments was attained through professional 
consensus on the wording of each item in relation to 
the attributes of effective schools. Also, predictive 
validity'^ was determined by how well survey 
responses were related to student achievement 
(reading, math and writing scores) in the pilot 
schools. 

Both reliability and predictive validity estimates 
were made by using data from the four highest 
achieving and four lowest achieving pilot schools. 
The idea was to calibrate the survey instruments to 
provide maximum distinction between groups of 
schools based on student achievement. Once the 
technical properties of the surveys were determined, 
then results could be analyzed for all pilot project 
schools in both higher and lower achieving groups. 



'2 Cftmbach's alpha was used to estimate the sUtistical 
reliability of the surveys. 

13 Correlations were used to estimate predictive validity. 
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The following format is used whenever overall 
results are presented: the eight higher achieving 
pilot schools are referred to as Group "A" and the 
eight lower achieving pilot schools are referred to as 
Group "B". 

What were the results of the validation 
phase? 

Chapter 1 evaluators in each participating district 
took the staff surveys to their respective school sites, 
monitored the administration procedures, and 
gathered all surveys upon completion. A total of 
829 staff responses were eventually returned for 
analysis. The next figure shows who filled out the 
survey. 

Staff Survey Respondents 



P:rc«nt of RMpondtnU 




Of the total respondents, 560 (or 68%) were 
teachers, the vast majority of whom were assigned to 
regular classrooms. There were 258 support staff 
(or 31%), including aides, tutors, counselors and 
others. There were only 1 1 (or 1 %) administrators. 
Clearly, teachers had the strongest voice in 
providing input on the staff survey. 

Reducing the length of the original staff survey (117 
items) was one of the first issues addressed. Items 
were eliminated from the survey based on a number 
of considerations, including the percent of missing 
or "Don't Know" responses, correlations with 
student achievement, and professional judgment. 
The end result was a 74 item staff survey that was 



both reliable^* and valid. In its final form, the 
staff survey had seven subscales, each with five to 
thirteen items to represent correlates of effective 
schools. 

Wiat methods were used to compare higher 
and lower achieving pUot schools? 

T-tests were used to determine how well the survey 
actually distinguished between higher achieving 
(Group "A") and lower achieving (Group "B") 
schools. Statistically significant differences were 
found for all subscales and the total; however, the 
practical significance of these differences varied 
somewhat. Appreciable effect sizes were found for 
five of the subscales plus the total score on the 
survey. 

In the following discussion, symbols are used to help 
distinguish among variables with different effect 
sizes : ♦ for large, ❖ for medium, for small, 
and D for marginal, but statistically significant. 
Also, the results from each section of the staff 
survey have been placed on a common scale that can 
be expected to range between one and ten with an 
average value of five and a standard deviation of 
two.'^ A common scale was needed to facilitate 
comparisons between correlates because each one 
had a different number of items. The total score for 
the staff survey was obtained by simply adding all 
seven subscales. 

What were the results? 




In general, the practical 
significance, (effect size) of 
differences between higher and 
lower achieving schools on the 
staff survey was not as 



Reliability estimates (alpha) ranged from .98 for the Total 
score to .84 for the Monitoring subscale. A rule of thumb used 
for "adequate" reliability was .75 or higher. 
'5 Correlations with student achievement ranged from .55 for 
the Safe and Orderly subscale to .32 for the Home School 
Relations subscale. Correlations of .30 and above are 
considered to have at least medium effect sizes. 
>* Scaling formuhi: ((raw score - subscale mean) / (subscale 
standard deviation/2)) 4- 5. 
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pronounced as some of the data base variables 
previously examined (e.g., poverty level, minority 
students). However, the sample was somewhat 
limited (16 schools in the pilot project) and the 
primary purpose was to field-test the instruments, 
not to make definitive statements about correlates of 
effective schools. Even so, the field-test results are 
instructive. They show that a paper and pencil 
survey can detect differences between higher and 
lower achieving schools, that all differences between 
groups of schools had statistical significance and that 
most had practical significance, as well. More than 
anything else, the results of the field-test offer 
encouragement for supporting, planning and 
improvement efforts based on the feedback of a 
school's most valuable resource-its staff. 



Differences Between Pilot School Groups: 
"Effective" School Correlates 




0 5 10 

Scale Value 

Common scale: Average=5; Std Dev=2 



As shown in the figure above, there were three parts 
of the staff survey that provided the greatest contrast 
between groups of higher and lower achieving 
schools. 

❖ Nine items grouped into Safe and Orderly 
Environment had the highest correlation with student 
achievement, provided the sharpest contrast between 
higher and lower achieving schools, and had the 
largest effect size of all the correlates. The 
following are the two items found to be the most 
strongly related to achievement: 

Staff members are treated respectfully by 
students. 

Vandalism or destruction of school property by 
studen'is is ffiH a problem. 
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The eleven items within the Instructional 
Leadership subscale had a moderate correlation with 
achievement and resulted in a substantial, but small 
effect size for differences between Group "A" versus 
Group "B" schools. The two items most strongly 
related to achievement were: 

The principal is highly visible, making frequent 
informal contact with students and teachers. 

The principal is accessible to discuss matters 
dealing with instruction. 



The subscale dealing with High Expectations 
contained eleven items and also had a moderate 
correlation with achievement. The effect size for 
differences between higher and lower achieving 
schools was also substantial, but slightly smaller. 
The two items most strongly related to achievement 
were: 

There is a positive school spirit. 

Students try to succeed in their classes. 

The figure below provides graphic information about 
the remainder of the staff survey. While differences 
between higher and lower achieving schools were 
not as pronounced as with the previous three 
correlates, all proved to be statistically significant 
and most had appreciable practical significance, as 
well. 

Differences Between Pilot School Groups: 
"Effective' School Correlates 



Homa/School Ralatiorw 



School Miwion 



Opportunity to Lmhi 



Fr«qu*nt Monitoring 
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Common actta: Avtrag*«S; Std 0«v>2 



<■ Home-School Relations consisted of thirteen 
items that were somewhat related to achievement. 
The practical significance estimated for this correlate 
was substantial, but relatively small. Top items in 
this area included: 

Most Parents rate this school as superior. 

Communication with parents is clear, effective, 
and frequent. 

The extent to which staff perceived their school 
as having a Clear Mission showed an appreciable, 
but relatively small, difference between Groups "A" 
and "B". Items most closely related to student 
achievement included: 

The school's mission statement is known and 
communicated clearly. 

The written statement of purpose defines 
academic goals that focus on student learning and 
achievement as this school's major 
responsibilities. 

□ Opportunity to Learn had 12 items which were 
moderately correlated with student achievement, but 
the effect size just missed the cutoff for practical 
significance. This finding does not diminish the 
importance of this area; rather, it suggests that other 
correlates reveal more of a difference. Items that 
did have the strongest relationship with achievement 
were: 

Students are engaged in learning activities from 
the beginning until the end of the instructional 
period. 



The time set aside for instruction is free from 
interruptions (e.g., intercom, messages, 
assemblies, mowing the lawn). 
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Q Frequent Monitoring of Student Progress 
consisted of five items which were moderately 
correlated with achievement, but had the lowest 
effect size of any subscale. Again, the statistical 
findings do not suggest that this correlate is 
unimportant, rather that differences among groups of 
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pilot schools were less disiinct. Top items in this 
category included: 

Teachers apply consistent criteria to assigning 
grades. 

The results of in-class assessments are used to 
examine students' strengths and weaknesses and 
to give feedback to students. 

Last, but certainly not least, the combination of 
all items produced an overall snapshot of staff 
perceptions that came the closest to Safe and Orderly 
Environment in terms of distinguishing between 
groups of schools. This fmding suggests that the 
overall assessment of a school on the basis of all 
seven effective school correlates may be as valuable 
as any of its individual parts. 

What conclusions can be drawn from the 
survey field-test? 

The "Successful Schools" staff survey was found to 
be quite reliable in terms of its internal consistency 
and at least moderately effective in differentiating 
between higher and lower achieving schools. It has 
the potential for supporting staff development, needs 
assessment, planning and evaluation functions 
associated wi^ both Chapter 1 Schoolwide Projects 
and the larger set of activities associated with local 
school improvement efforts. 

Results from the field-test administration confirm 
that violence and disruptive student behavior have 
become major concerns among school staff, in this 
case those working with children from low income 
families. This finding affirms, the most recent 
Gallup Poll of the Public's Attitudes Toward Public 
Schools (Elam, Rose, and Gallup, 1994) where: 

For the first time ever, the category "fighting, 
violence, and gangs" shares the number-one 
position with "lack of discipline" as the biggest 
problem confronting local public schools. 

InstTictional Leadership and High Expectations 
round out the trio of effective school correlates for 
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which there was the sharpest distinction between 
higher and lower achieving schools. The other parts 
of the survey, as well as the total score, also provide 
important information. 



Successful Schools Parent Survey 

Since the constructive involvement of parents is a 
key ingredient in making schools successful, it was 
only natural that the research team would want to 
include them in the development and field-testing of 
survey instruments. Using the Home-School 
Relations section of the staff survey as a foundation, 
the team modified certain items and added others in 
hopes of gaining the insight of parents. 

How was the parent survey validated? 

The Successful Schools parent survey consisted of 
23 items plus several questions about the person 
filling out the form. The survey was distributed via 
students at the same 16 pilot project schools where 
the staff survey was field-tested. Despite the fact 
that each school did not receive the same number of 
surveys and that they were distributed at different 
times, a total of 2,382 responses were eventually 
collected for analysis. Based on the number of 
students at each school, it was estimated that the 
return rate ranged between 1% and 66% and was 
about 20% overall. 

Similar procedures for determining the reliability 
and validity of the staff survey were applied to the 
parent version, but with quite different results. 
Based on data from the four highest achieving and 
four lowest achieving schools, it was found that the 
reliability (internal consistency) of the parent survey 
was quite adequate'"^. However, there was little 
indication of predictive validity in terms of the 
survey's relationship to either student achievement 
(reading, math and writing scores) or learning 
environment conditions (promotion, attendance and 
suspension rates) at the pilot project schools. This 
led to the conclusion that the parent survey, while 

''^ Reliability was estimated at .92, using Cronbach's alpha. 
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reliable, was actually measuring something different 
than originally intended. 

How well were individual schools 
represented in the survey data? 

Early on in the analysis, it was discovered that 
parents at some schools were much better 
represented than others. In fact, one school turned 
in so many surveys (836 or 35% of the total) that 
they tended to dominate all the others. Without 
some adjustment, this would have distorted the 
overall results and made comparisons between 
higher and lower achieving schools misleading. 

A decision was made to draw a random sample from 
the school that turned in so many parent surveys, so 
as to smooth out the representation of each school in 
ihe total data. The original number of 836 was 
reduced to 175, which still left that school in the 
position of having the most surveys in its group, but 
not to the point of dominating all others. 

What did parents say (or not say) about their 
schools? 

Having reconciled the proportionate number of 
surveys from each school as best as possible, the 
data were analyzed so that results of the field-test 
could be shared. One of the first things that became 
clear was th^X parents were often reluctant to answer 
many of the survey items. 

It was found that about 20% of the time, parents of 
students at the 16 pilot sites either did not respond or 
answered "Don't Know," even to factual statements 
such as the existence of a written discipline policy, a 
statement of purpose for the school, or an active 
parent group. There were five items for which one 
third or more of the parents at half or more of the 
pilot schools either did not respond or answered 
"Don't Know": 

A written statement of purpose exists for this 
school. 
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Parents are encouraged to share ideas for school 
improvement with administration and staff in this 
school. 

The parent organization of this school is 
considered important by the administration and 
teachers. 

My child's teachers spend more time 
communicating with me about the good things my 
child does than the bad. 

There is an active parent group at this school. 
Items for which there were large numbers of missing 
or "Don't Know" responses point to three areas of 
parental involvement with a low level of awareness 
or a general reluctance to respond. These items also 
had some of the lowest rates of agreement by parents 
who did respond. 

1. School mission or purpose 

2. Status of the local parent organization 

3. Positive feedback from/input to school 
improvement 

On the other hand, there was an equal number of 
items about which parents were not reluctant to 
respond and also had a high rate of agreement. The 
following are five items which had a relatively low 
incidence of missing or "Don't Know" responses 
(10% or fewer in two-thirds or more of the schools): 

I feel free to initiate contact with my child's 
teacher. 

I make an effort to be informed about my child's 
educational program. 

I cooperate with my child's teacher to see that 
homework is completed. 

My child completes assigned homework before 
going to school. 

I support the school homework policy. 

Items for which there were relatively low numbers 
of missing or "Don't Know" responses point to two 



areas of parental involvement where there appears to 
be a high level of awareness/understanding or a 
general willingness to respond, as well as a high 
level of general agreement. Incidentally, many of 
these represent things tliat parents are supposed to 
do. 

1. Communication with their child's teacher and 
awareness of their child's educational program 

2. Support of homework policy 



With few exceptions, when parents did respond, 
they tended to agree with almost every item on the 
survey. These results reflect well the public 
attitudes captured in recent years through the Annual 
Phi Delta Kappa Gallup Poll>8. 

• People gave the school attended by their eldest 
child good grades ~ 70% gave it an A or a B, and 
92% gave it a passing grade. But they continue to 
give the nation's schools considerably lower grades: 
only 22% award the nation's schools an A or a B, 
while 49% give them a C. 

The parent survey data bear a striking resemblance 
to the poll. When asked whether they agreed or 
disagreed with the statement--"! would rate this 
school as superior''~parents in both higher and 
lower achieving schools responded affirmatively; 
76% in Group "A" and 72% in Group "B". 

Yfere there anx items that distinguished 
between higher and lower achieving 
schools? 

While parents were overwhelmingly positive when 
they did respond, there was one item that showed a 
difference of at least 10% between parent responses 
in higher versus lower achieving schools: 

I am aware of a written discipline policy. 

It is important to note that the one item that most 
distinguished between higher and lower achieving 



schools dealt with discipline. This finding very 
strongly reinforces the fact that the Safe and Orderly 
correlate on the staff survey was the area that 
provided the greatest distinction between higher and 
lower achieving schools. 



Parent Comments 

In addition to the 23 survey items, parents were 
afforded the opportunity to make written comments 
in three areas: 

1. What is the school doing that is most helpful 
for your child? 

2. Please tell us what the school could do to help 
your child. 

3. Please provide other comments. 

Over 1,200 written comments were received in the 
first area, over 1,000 in the second, and about 600 
in the third. It should be noted that this count is for 
comments, since the same individual often provided 
multiple responses. Comments were coded as they 
related to the seven correlates of effective schools, 
then the data were analyzed by higher versus lower 
achieving groups of pilot project schools. 

What did parents say was most helpful for 
their children? 

Like their responses to the survey items, parents in 
both high and low achieving schools were generally 
positive about the help provided to their children. 
Almost two-thirds of the parents in both groups 
indicated that providing an Opportunity to Learn was 
most helpful for their children and that effective 
teachers and teaching strategies were the keys. A 
significantly higher percentage of parents in higher 
performing schools indicated that special programs 
(such as Chapter 1) were helpful to their children. 



"Elam, Rose and Gallup (1994) 
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What did parents say schools could do to 
help their children? 

Parents at lower performing schools were 
significantly^' more likely to indicate that the school 
should provide a Safe and Orderly environment for 
their child than parents at higher performing schools. 
The comments indicated a need for more effective 
discipline, safer facilities, respect for all students, 
and improved student behavior. Also, a 
significantly greater percentage of parents at lower 
performing schools indicated that more homework 
should be assigned or that the homework policy 
should be improved. 

Were there other comments? 

On their own, parents in lower performing schools 
were significantly more likely to comment 
negatively on safety and orderliness at their schools. 
Comments like the following reflected a general call 
for more discipline: "Teachers are handcuffed when 
dealing with discipline. Rules need to be enforced." 
Also, a significantly greater proportion of parents 
commented positively about teachers and teaching 
strategies at higher performing schools (19%) than at 
lower performing schools (8%). 

What conclusions can be drawn from the 
parent surveys? 

The 23 item parent survey was found to be reliable, 
but of very limited use for distinguishing between 
higher and lower achieving schools. The responses 
obtained in the pilot project field-test were 
characterized by a relatively high proportion of 
missing and "Don't Know" answers, especially to 
items that asked about their school's written 
statement of purpose, the status of the local parent 
group, and the school's receptivity to ideas for 
school improvement. 

When parents did respond, they tended to be in 
general agreement with most items. Like the results 



Z-test for the difTereoce between two population proportions 
(Hinkle, Wiersma, and Jurs, 1979) 
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from the most recent Gallup poll on public attitudes, 
parents gave high ratings to the school their child 
attended, regardless of its achievement level. The 
only item that showed a clear distinction between 
high and low achieving groups of schools had to do 
with parents' awareness of a written discipline 
policy. This finding lends support to the importance 
of the Safe and Orderly correlate of effective 
schools. 

Analysis of written comments confirmed parents' 
tendency to be generally positive and appreciative of 
the teaching staff. Their recommendations, 
especially at lower performing schools, called for 
improved safety and better discipline among 
students. This finding also reinforces the Safe and 
Orderly aspect of schools as having an important 
bearing on student achievement. 



The last section of this report deals with the site 
visits that were conducted at each of the 16 pilot 
project schools. During their short stay on the 
campus of each school, teams of four professional 
educators spent most of their time interviewing the 
principal, a number of classroom teachers, various 
support staff, and parents. The teams also collected 
resource documents, took photographs, and made 
general observations about physical conditions at the 
school. An important activity of the teams was to 
summarize their visit and general impressions. 

What was the composition and structure of 
the on-site teams? 

In May of 1994, each of the 16 pilot project schools 
was visited by a four member team which spent one 
full day on campus. None of the team members 
were employed by or directly involved with the 
district in which their assigned schools were located. 
A total of 29 different people participated as on-site 
team members, including 15 school district 
personnel, eight from the State Department of 
Education, four from the Department's regional 
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offices, and two from the Technical Assistance 
Center in Atlanta. One person at the state leveli put 
together all the visiting teams, was in charge of 
scheduling, and was on call to resolve any questions 
or changes. Each team had a designated leader, who 
was a Chj^ter 1 evaluator or coordinator from a 
different school district, and who took charge of 
local arrangements, team orientation, and the 
completion/return of data collection forms. 

What was done during the on-site visits? 

The major activity of the on-site teams was to 
accomplish a rigorous interview schedule which 
included the principal, about six classroom teachers, 
two instructional aides, the guidance counselor, the 
head custodian, and two parents. Persons to be 
interviewed were identified by the school principal, 
which had both advantages and disadvantages. On 
the one hand, allowing the principal to designate 
respondents helped minimize the intrusive nature of 
the visits and any resemblance to a compliance audit, 
which was not the purpose. On the other hand, the 
teachers and parents chosen to be interviewed were 
likely hand picked to reflect the most positive 
aspects of the school. 

The average length of time for these interviews was 
about 45 minutes, except for the district Chapter 1 
coordinator who was interviewed separately by the 
team leader for a considerably longer period of time. 

Interview questions were developed in advance of 
the school visits and were tailored for each type of 
respondent. In general, the questions were framed 
around the seven correlates of effective schools and 
were intended to gather more in-depth information 
than was available through the staff and parent 
surveys. There was, however, one unique question 
that was asked at the end of each interview: Would 
you want your child to attend this school? 

Interview questions were preprinted on different 
color forms with a purposefully limited writing 
space (about 3") provided after each question. The 
interview procedure called for the on-site teams to 
split into pairs so that they could alternate asking a 
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question, maintaining eye contact, listening, 
probing, etc. while their team member concentrated 
on recording and summarizing the responses. The 
two people conducting the interviews switched roles 
as each question was completed. When all 
interviews had been completed, the teams 
reassembled for a wrap-up session to share and 
summarize their impressions. 

Did the general impressions of the on-site 
teams reveal differences between schools? 

By design, the on-site teams typically visited one 
higher achieving and one lower achieving school in a 
district. Team members who visited both types of 
schools were in near universal accord that the 
differences were obvious and detectable after just a 
short time on campus, though they were not advised 
beforehand. The following are some of ihe major 
differences as captured in the summaries written by 
on-site teams: 

♦ They described discipline in a positive way 
for higher achieving schools more often than for 
lower achieving schools. 

♦ They discussed the principal's leadership in a 
positive way more often for higher achieving 
schools. 

♦ They discussed staff development activities in 
high achieving schools, but not in low achieving 
schools. 

♦ They discussed "change" four times as often 
in low achieving schools. 

♦ Teams frequently described high achieving 
schools as having a unified staff. 

How were the interview responses analyzed? 



When all 16 pilot project 
schools had been visited, 
more than 200 interview 
forms had been 
completed, which resulted 
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in over 250 pages of transcribed notes. Given such 
a volume, qualitative analysis on this kind of open- 
ended data represented a huge challenge. A 
computer program^o that could help analyze the text 
from the interviews was purchased; however, each 
recorded comment still had to be coded and grouped 
by hand. Whenever possible, interview questions 
were grouped to match appropriate effective school 
correlates. 

What conclusions were drawn from the 
interviews? 

Despite the huge volume of 
1^^^^^ information that was 
j^H^^m gathered during the 

JHJI^p^ interview process, interview 
l^^^r"^ responses in general were 
not notably different in high 
versus low achieving schools. There were, 
however, subtle differences that did emerge. 

Instructional Leadership styles differed in the very 
highest and very lowest achieving schools. 
Principals in the first group apparently embraced 
shared decision-making, valued their teachers' input 
and also encouraged risk-taking and creativity. 
Principals at the lower achieving school asked for 
teacher input by and large after curriculum decisions 
had been made. 

Lower achieving schools articulated their school 
mission differently than higher achieving schools. 
The latter focused on their students as the main 
reason for existence. In lower achieving schools, 
there was no common thread as to the school's 
mission. 

In the area of Safe and Orderly environment, there 
was little difference among personnel who were 
interviewed. However, both groups identified 
students with severe emotional and/or family 
problems as those who presented the most discipline 
problems. 



20 QUALPRO, ft text-data base manager was used to analyze 
interview notes that were transcribed to WordPerfect files. 
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What could be done differently during the 
on-site visitation phase? 

For all the expense, data collection activity, and 
analysis, the on-site visitation phase of the 
Successful Schools project did not yield a 
comparable amount of information. While team 
members were unanimous as to the value of the on- 
site visits, it would appear that different activities 
should be involved. 

Instead of conducting numerous interviews based on 
questions that can be more economically collected 
via staff surveys, perhaps there are more productive 
avenues of inquiry. It would be instructive to 
follow-up on several areas of the data base analysis 
where more information would help clarify 
significant, but incomplete findings. 

For example, among high poverty schools, it was 
found that higher achieving ones have more support 
staff per SOO students than do lower achieving 
schools. On-site visits might probe more deeply to 
learn what kinds of support staff are involved, how 
their activities support instruction and/or student 
learning, and what kinds of qualifications and 
training activities distinguish support staff in higher 
versus lower achieving schools. 

Members of on-site observation teams might also 
examine the issue of teacher turn-over in high 
poverty schools as well as probe into differences in 
the planning, organization and application of staff 
development activities. Since high poverty schools 
have more first year teachers, there may be 
circumstances or personal characteristics which 
predispose some to have better experiences (and 
contribute more) than others. 

Given the results of the staff and parent surveys, it 
would seem natural that part of the on-site visits 
should focus on student discipline and safety. How 
are discipline policies developed, implemented and 
supported? Are there notable strategies for 
promoting a safe and orderly environment in schools 
that would otherwise be considered high risk? 



What do principals do that make them instructional 
leaders in the eyes of teachers, parents and students? 
How do district policies affecting accountability and 
assignment affect the placement of principals at high 
poverty schools? 

Analysis of financial data suggest that the 
availability of more money, especially through 
Chapter 1 Schoolwide Projects, does not necessarily 
result in higher achieving schools. How do 
successful Schoolwide Projects spend their extra 
resources? How do they maintain their focus on the 
academic needs of their students? How are 
successful Schoolwide Projects coordinated with 
school improvement plans? 

In addition to these follow-up topics, the on-site 
visits might also explore curriculum and instructional 
aspects of successful schools. Are there specific 
instructional strategies that are used in higher versus 
lower achieving schools? Does the arrangement of 
instruction have common features in high achieving 
schools? What about materials and/or equipment? 
How do differences in school facilities (design and 
condition) contribute to student achievement? 




All activities were ultimately directed at answering 
the same fundamental question: What distinguishes 
successful, high poverty schools from similar but 
lower achieving schools? 

Data Base Analysis, The first section on the impact 
of poverty clearly showed that schools with high 
concentrations of students from low income families 
are likely to be low achieving schools which also 
have lower promotion and higher suspension rates. 
High poverty schools typically serve considerably 
more minority students and experience more student 
turnover than medium or low poverty schools. 
Contrary to popular notions, high poverty schools (at 
least in Florida) have more money to spend on 
students than lower poverty schools in the same 
district. High poverty schools typically have more 
than their share of first year teachers and fewer 
experienced teachers, but also tend to have 
considerably more support personnel. 

Analysis of conditions associated with higher 
achievement pointed to few things that can be 
readily altered. The relative concentration of poor 
students, even among high poverty schools, emerged 
as the strongest predictor~the more students from 
low income families, the lower the school's 
achievement level. The fact that higher levels of 
Federal expenditures were not associated with higher 
achievement raised questions about the effective use 
of extra resources available though Chapter 1 
Schoolwide Projects. The positive relationship 
between more support staff and higher reading 
achievement is encouraging, but not well enough 
defined to suggest what action should be taken. 



The Chapter 1 Successful Schools study consisted of 
three distinct research perspectives: 

• Data base analysis, using Florida School Reports 
and related information, 

• Staff and parent surveys based on effective 
school correlates, and 

• On-site observations and interviews. 
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The most encouragement came from the section on 
"successful," high poverty schools. Despite the 
tremendous odds against them, some high poverty 
schools actually do better than state averages and 
even better than some low poverty schools. Twenty- 
nine of the highest achieving schools were compared 
with an equal number of the lowest achieving 
schools-all of which were high poverty. While 
achievement differences were dramatic, the two 
groups of schools were not distinguishable in a 
number of areas: promotion, attendance, LEP, 
mobility, class size, regular program expenditures, 

32 



or teachers with advanced degrees. The successful 
group of schools was distinguishable in terms of 
having fewer minority students, more support staff 
and lower suspension rates. What may work for 
successful, high poverty schools may not work for 
all high poverty, high minority schools. This 
preliminary study suggests that educators need to 
explore methodologies more clearly attuned to 
specific student populations. 

Some high poverty, high minority schools in this 
pilot study have, in fact, been successful despite 
student population characteristics which otherwise 
have shown low achievement. Further study is 
required to determine if the methodologies and 
characteristics of these successful schools can be 
identified and transferred to other high poverty, high 
minority schools. 

Stqff and Parent Surveys. This section of the report 
concentrated on a relatively small number of high 
poverty schools in order to field-test survey 
instruments based on "effective" school correlates. 
Instruments used with school staff members were 
shown to be reliable and valid and the preliminary 
result'i pointed to significant differences between 
higher versus lower achieving schools. Of the seven 
characteristics of effective schools that were used in 
the staff survey, three areas emerged as being the 
most prominent: Safe and Orderly Learning 
Environment, Instructional Leadership and High 
Expectations for students. 

Results from the parent survey^were quite different. 
While the instrument itself proved reliable, there 
was little distinction between the responses of 
parents whose children attended higher versus lower 
achieving schools-both groups were very positive. 
Also many parents often did not respond or 
answered "Don't Know" to many of the survey 



items, leaving some doubt as to the preliminary 
results. On the other hand, it was quite interesting 
that the only question that clearly distinguished 
between higher and lower achieving schools had to 
do with discipline. 

Comments which parents wrote down added another 
dimension to the survey results. While they were 
generally positive overall, parents' recommendations 
called for improved safety and better student 
discipline, especially at lower achieving schools. 

On-Site Observations and Interviews. The final 
phase of the Successful Schools pilot project 
culminated with site visits at the same 16 schools 
where staff and parent surveys were administered. 
While all team members who participated strongly 
endorsed the concept of site visits, the types of 
activities will likely change in the future. For all the 
data collected, only a few distinguishing features 
between higher and lower achieving schools were 
discovered, primarily subtle differences in leadership 
styles and focus of the school's mission. 

Instead of interviewing staff and parents using 
questions similar to the survey instruments, future 
on-site visits will likely concentrate on following up 
on some of the research findings that need further 
clarification, such as contribution of support staff, 
strategic use of federal funds, better understanding 
of effective programs to ensure school safety and 
discipline, and exploration of curriculum/instruction 
issues. 

Like all good research activities, the Chapter 1 
Successful Schools Study contributes to our existing 
knowledge base by confirming some relationships 
between school conditions and student achievement, 
challenging others, and raising even more questions 
to be addressed. 
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